Global Newton Method for stochastic games

The Global Newton Method for games in normal form and in extensive form is shown to have a natural extension to computing Markov-perfect equilibria of stochastic games. GLOBAL NEWTON METHOD FOR STOCHASTIC GAMES SRIHARI GOVINDAN AND ROBERT WILSON Abstract. The Global Newton Method for games in normal form and in extensive form is shown to have a natural extension to computing Markov-perfect equilibria of stochastic games. The Global Newton Method for games in normal form and in extensive form is shown to have a natural extension to computing Markov-perfect equilibria of stochastic games. In [4] we implement the Global Newton Method to compute Nash equilibria of N-player games in normal form, and in [3], to compute ε-perfect equilibria of N-player games in extensive form. These results are extended and applied by Blum, Shelton, and Koller [1], who also report more substantial computational experience than we report in [4]. In [5] we implement a discrete version based on Kuhn’s triangulation of the strategy space and report computational experience. In this paper we adapt the Global Newton Method to compute Markov-perfect equilibria of stochastic games, which are important in empirical studies of oligopolistic industries, as in Doraszelski and Satterthwaite [2], Herings and Peeters [6, 7], and Pakes and McGuire [9, 10]. Our key tool is an extension of the structure theorem, established by Kohlberg and Mertens [8] for Nash equilibria of static games in normal form, to Markov-perfect equilibria of stochastic games played over an infinite number of stages. Section 1 reviews briefly how the structure theorem for Nash equilibria of N-player games enables the Global Newton Method [GNM hereafter] to compute equilibria. Section 2 establishes the analogous structure theorem for Markov-perfect equilibria of stochastic games. Section 3 then derives the GNM trajectory that passes through an odd number of Markovperfect equilibria. The resulting algorithm is essentially identical to the one presented in detail in [4] other than accounting for transitions among states. 1. Brief Summary of the Global Newton Method Consider a game in normal form with N players for which each player n has a set Sn of pure strategies, and a simplex Σn = ∆(Sn) of mixed strategies. Define the product sets S = ∏ n Sn and Σ = ∏ n Σn of profiles of pure and mixed strategies, and the disjoint union Date: March 2007. Revised February 3, 2008.